Volumetric tracking for orthogonal displays in an electronic collaboration setting

ABSTRACT

Systems and methods for providing volumetric tracing for orthogonal displays in an electronic collaboration setting. In some embodiments, an Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory including program instructions stored thereon that, upon execution by the processor, cause the IHS to: identify a first characteristic associated with a given one of a plurality of participants of an electronic collaboration session, at least in part, using an image sensing device; identify a second characteristic associated with the given participant, at least in part, using a proximity sensing device; identify a volumetric space for the given participant based upon a combination of the first and second characteristics; and provide a user interface to the given participant. The user interface may have a set or one or more features selected, at least in part, based upon a characteristic of the volumetric space.

FIELD

The present disclosure generally relates to Information Handling Systems(IHSs), and, more particularly, to systems and methods for providingvolumetric tracing for orthogonal displays in an electroniccollaboration setting.

BACKGROUND

As the value and use of information continues to increase, individualsand businesses seek additional ways to process and store information.One option is an Information Handling System (IHS). An IHS generallyprocesses, compiles, stores, and/or communicates information or data forbusiness, personal, or other purposes. Because technology andinformation handling needs and requirements may vary between differentapplications, IHSs may also vary regarding what information is handled,how the information is handled, how much information is processed,stored, or communicated, and how quickly and efficiently the informationmay be processed, stored, or communicated. The variations in IHSs allowfor IHSs to be general or configured for a specific user or specific usesuch as financial transaction processing, airline reservations,enterprise data storage, global communications, etc. In addition, IHSsmay include a variety of hardware and software components that may beconfigured to process, store, and communicate information and mayinclude one or more computer systems, data storage systems, andnetworking systems.

For purposes of this disclosure, an IHS may include any instrumentalityor aggregate of instrumentalities operable to compute, calculate,determine, classify, process, transmit, receive, retrieve, originate,switch, store, display, communicate, manifest, detect, record,reproduce, handle, or utilize any form of information, intelligence, ordata for business, scientific, control, or other purposes. For example,an IHS may be a personal computer (e.g., desktop or laptop), tabletcomputer, mobile device (e.g., personal digital assistant (PDA) or smartphone), server (e.g., blade server or rack server), a network storagedevice, or any other suitable device and may vary in size, shape,performance, functionality, and price. The information handling systemmay include Random Access Memory (RAM), one or more processing resourcessuch as a Central Processing Unit (CPU) or hardware or software controllogic, ROM, and/or other types of nonvolatile memory. Additionalcomponents of the information handling system may include one or moredisk drives, one or more network ports for communicating with externaldevices as well as various Input and Output (I/O) devices, such as akeyboard, a mouse, touchscreen and/or a video display. The IHS may alsoinclude one or more buses operable to transmit communications betweenthe various hardware components. Further, the IHS may includetelecommunication, network communication, and video communicationcapabilities.

SUMMARY

Embodiments of systems and methods for providing volumetric tracing fororthogonal displays in an electronic collaboration setting aredescribed. In an illustrative, non-limiting embodiment, an InformationHandling System (IHS) comprises one or more processors and a memorycoupled to the one or more processors, the memory including programinstructions stored thereon that, upon execution by the one or moreprocessors, cause the IHS to: identify a first characteristic associatedwith a given one of a plurality of participants of an electroniccollaboration session, at least in part, using an image sensing device;identify a second characteristic associated with the given participant,at least in part, using a proximity sensing device; identify avolumetric space for the given participant based upon a combination ofthe first and second characteristics; and provide a user interface tothe given participant, wherein the user interface has a set or one ormore features selected, at least in part, based upon a characteristic ofthe volumetric space.

For example, the image sensing device may include a three-dimensional(3D) camera and the proximity sensing device may include a millimeterRadio Frequency (RF) wave or ultrasonic sensor array.

The first characteristic may include at least one of: a position of thegiven participant in three-dimensional space, an identity of theparticipant, a size or reach of the participant, a role of theparticipant in the electronic collaboration session, a gesture made bythe given participant, or an intent of the given participant, the secondcharacteristic may include a position of the given participant relativeto another participant, and the user interface may include at least oneof: a Graphical User Interface (GUI), a speech recognition interface, ora gesture recognition interface.

In some cases, to identify the volumetric space, the programinstructions, upon execution, may cause the IHS to determine athree-dimensional area around the given participant from within whichgesture or voice commands issued by the given participant arerecognizable by the IHS, and outside of which gesture or voice commandsissued by the given participant are not recognizable by the IHS.

The characteristic of the volumetric space may include at least one of:a size, a position, or a shape of the volumetric space. To identify thevolumetric space, the program instructions, upon execution, may furthercause the IHS to: determine that another participant moved closer to thevolumetric space; and reduce the volumetric space to disable one or moregestures available to the participant prior the other participant havingmoved. Additionally or alternatively, to identify the volumetric space,the program instructions, upon execution, may further cause the IHS todetermine that another participant moved farther from the volumetricspace and increase the volumetric space to enable one or more gesturesunavailable to the participant prior the other participant having moved.

In some embodiments, the IHS may be configured to identify anothervolumetric space for another participant and provide another userinterface to the other participant, wherein other user interface has asecond set of features accessible to the other participant to theexclusion of the given participant. For example, other participant maybe disposed behind the participant with respect to the image capturedevice. The IHS may receive two or more commands and determine that agiven command was issued by the given participant based upon it havingbeen detected as originated from the volumetric space and that anothercommand was issued by the other participant based upon it having beendetected as originated from the other volumetric space, at least inpart, using the proximity sensing device.

In some cases, a first set or one or more features accessible to thegiven participant to the exclusion of the other participant may includea GUI, and the second set of features accessible to the otherparticipant to the exclusion of the given participant may include avoice or gesture recognition interface. Additionally or alternatively,the first set or one or more features accessible to the givenparticipant to the exclusion of the other participant may include agesture recognition interface, and the second set of features accessibleto the other participant to the exclusion of the given participant mayinclude a voice recognition interface. Additionally or alternatively, afirst set or one or more features accessible to the given participant tothe exclusion of the other participant may include a touch interface,and the second set of features accessible to the other participant tothe exclusion of the given participant may include a gesture recognitioninterface.

In various embodiments, the IHS may be configured to identify adifferent volumetric space for the given participant based upon a changein at least one of the first or second characteristics during theelectronic collaboration session and provide another user interfacehaving another set of one or more features selected, at least in part,based upon a characteristic of the different volumetric space.

For example, in response to the characteristic of the differentvolumetric space indicating that the given participant's role in theelectronic collaboration session has increased, the other set offeatures may be larger than the set of features, and in response to thecharacteristic of the different volumetric space indicating that thegiven participant's role in the electronic collaboration session hasdecreased, the other set of features may be smaller than the set offeatures. Additionally or alternatively, in response to thecharacteristic of the different volumetric space indicating that thegiven participant has moved closer to a display, the other set offeatures may be larger than the set of features, and in response to thecharacteristic of the different volumetric space indicating that thegiven participant has move farther from the display, the other set offeatures may be smaller than the set of features.

The electronic collaboration session may use a first screen as part of ahorizontal display device and a second display as part of a verticaldisplay device. The user interface may be dynamically distributedbetween the first and second displays based upon the characteristic ofthe volumetric space.

In another illustrative, non-limiting embodiment, a method may implementone or more of the aforementioned operations. In yet anotherillustrative, non-limiting embodiment, a memory device may have programinstructions stored thereon that, upon execution by an IHS, cause theIHS to perform one or more of the aforementioned operations.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention(s) is/are illustrated by way of example and is/arenot limited by the accompanying figures. Elements in the figures areillustrated for simplicity and clarity, and have not necessarily beendrawn to scale.

FIG. 1 is a diagram illustrating an example of an environment wheresystems and methods for providing volumetric tracing for orthogonaldisplays in an electronic collaboration setting may be implementedaccording to some embodiments.

FIG. 2 is a diagram of an example of an Information Handling System(IHS) configured to provide volumetric tracing for orthogonal displaysin an electronic collaboration setting according to some embodiments.

FIG. 3 is a diagram of an example of a virtual meeting system accordingto some embodiments.

FIG. 4 is a diagram of an example of a gesture/intent detection systemaccording to some embodiments.

FIG. 5 is a diagram of an example of a proximity detection systemaccording to some embodiments.

FIG. 6 is a diagram of an example of a switching system according tosome embodiments.

FIG. 7 is a flowchart of an example of method for providing volumetrictracing for orthogonal displays according to some embodiments.

DETAILED DESCRIPTION

To facilitate explanation of the various systems and methods discussedherein, the following description has been split into sections. Itshould be noted, however, that the various sections, headings, andsubheadings used herein are for organizational purposes only, and arenot meant to limit or otherwise modify the scope of the description orthe claims.

Overview

The terms “electronic collaboration,” “electronic meeting,” “electroniccollaboration session,” “virtual collaboration session,” and “virtualmeeting,” as used herein, generally refer to interactions where somedegree or form of communication between participants makes use of anInformation Handling System (IHS). Modern tools that facilitate“electronic collaboration” include, but are not limited to, video anddata conferencing, instant messaging, electronic file sharing, and thelike. In the case of video and data conferencing, for example, local andremote participants (in different geographic locations) may join avirtual meeting from their individual IHSs as if they were physically inthe same room.

During an electronic collaboration session, a digital or virtualworkspace may offer in-room and/or remote participants the ability tointeract in real-time. In addition, one or more IHSs may provide, amongother things, meeting setup, participant identification, proximityawareness, volumetric gesture and intent analysis, display control,etc., as well as information management, indexing and search, and/ormeeting continuity. Moreover, a typical set of client capabilitiesworking in concert across potentially disparate devices may include:access to a common shared workspace with public and private workspacesfor file sharing and real-time collaboration, advanced digitalwhiteboarding with natural input to dynamically control access, robustsearch functionality to review past work, and/or the ability toseamlessly moderate content flow, authorization, and intelligentinformation retrieval.

In various embodiments, multiple displays may be shared and used at eachlocation or room involved in an electronic collaboration session. Theseshared display devices may convey, among other things, an image ofpresenter showing graphical information to other participants, as wellas the graphical information itself.

In some scenarios, at least two displays may be used in the samephysical room: a first display (e.g., vertically mounted on a wall) anda second display (e.g., part of a horizontal smart desk or table), in anorthogonal configuration with respect to each other. Although certaintechniques described herein are particularly well-suited for thosescenarios, a person of ordinary skill in the art will recognize thatthese techniques may also be applied to other configurations.

When certain aspects of the framework described herein are applied to anIHS, for instance, the IHS may become a fixed point of referenceproviding an individualized user interface to one or more participantsof the electronic meeting, potentially across multiple distinctdisplays. The IHS may maintain a relationship to a conference room andassociated resources (e.g., peripheral hardware), for example. Thisallows the IHS to be a central hub for organizing meetings, and it doesnot rely on a host participant and their device to be present.

To illustrate but a few of the many features that may be implementedusing systems and methods described herein, consider the following:

1. Multi-user interface and scaling. As the number of actions andreactions in a given electronic meeting grows, it quickly becomesdifficult to follow and gain an understanding of its general flow.Accordingly, in some cases, sessions may be moderated by an IHS thatrecords, analyzes, and determines a number of participants in the room,identities or one or more participants, identities which participantsare active and which are not, and/or identities any participant'sgestures and/or intent. Techniques described herein may provide aconversational user interface applicable to groups and communalenvironments, and may create injection points for the interface. Forexample, using exploration and comparative analysis of gesture patternsand voice (e.g., biometric feature extraction, analysis and selection),an IHS may be capable of determining “the most common and similar querypattern expressed among participants” or “how much variation is observedin a given gesture.” In some cases, multi-camera -based pose estimationmay be used. Pose states may be analyzed as a result of a pose trainingprocedure during which a set of representative human poses and/or motionsequences are collected as classified. Each motion pose may beclassified as belonging to a pose state, and motion data may beportioned into k segments with respect to a frame.

2. Communal interface policies. A policy may be implemented that isloosely interactive when two or more speakers are talking so that agraphical user interface follows one or more participants around theroom. Gesture elicitation studies collect natural and common trends inhuman gestures by categorizing similar gestures into a set of groupsbased on perception, background knowledge, and interaction contexts.These studies may involve, for example, visualizing multiple motion datasequences to support exploration and understanding of gesture motionpatterns, selecting interesting gesture patterns for detailed analysis,comparing gestures to identify similar and dissimilar gestures,supporting users to reflect their domain knowledge and interactioncontexts into aggregating similar gesture data, and generating patternanalysis results in a form so that findings and insights can be sharedwith other researchers and domains. In some cases, systems and methodsdescribed herein may enable manipulation of content based participant'sintent. For instance, if a participant leans over or talks, the IHS maymove a GUI toward that participant or it may expand the GUI based onprimary file content. A switch display camera may be used inasynchronous mode to track motion and position, which may be furtheraugmented using audio, device cues (e.g., pen or pointer), communicationsignals (e.g., WiFi), and/or proximity detection (e.g., millimeter RFsensors).

3. Obstacle handling in a communal interface for interactive andnon-interactive devices. Objects are be handled in a communal interface.For example, “virtual objects” are graphical elements drawn on adisplay, and include the target as well as a set of distracting objectsthat have a similar visual appearance as the target. Considerationstaken into account when handling objects using the system and methodsdescribed herein may include, but are not limited to: space consumption(amount of space utilized by the technique in addition to the spaceutilized by the physical objects themselves), visual clutter (degree ofvisual clutter produced, for instance, with occluded objects no longervisible), spatial transformation (impact on the geometric properties ofthe visual space, including distortion, displacement, and rotation), andintention support (intentional occlusion when a participant wants tointentionally hide virtual objects using physical objects).

4. Volumetric tracing from 2D to 3D for orthogonal display andcalibration. In some embodiments, interaction tracking may be enabledfor orientation of GUI and information relative to two orthogonaldisplays, for example. Techniques described herein may scale proximitysensing with a 3D camera in state sharing or screen shot mode, create anorthogonal touch space using touch fusion, and create a volumetric spacefor gesture and switch between touch and gesture. Moreover, volumetrictracing may allow an IHS to distinguish between certain participants'spheres of activity (e.g., when one participant is behind or nearanother participant in the same room).

5. Multi-device content alignment using touch and voice. In some cases,a laptop (e.g., a primary device in a smart desk and wall computer mightwant to share file and yet need to work in unison for data scaling). Inthose cases, a 3D camera may be used for GUI zoning for a remote usercamera and intended party. The IHS may follow the voice for filesharing, and/or it may adjust GUI based on size of the participant orthe participant's reach.

6. Tagging data based on data state relation to a participant(multi-user aware). In some cases, an IHS may use voice and touchfrequency to identify intent state and tag data or provision relateddata using a related data icon. It may use hand speed and frequency toidentify a most interacting participant, for example. It may also usemultiple gaze or blinking patterns to tag data of interest to aparticipant.

7. Audio-video synchronization system for orthogonal display systems. Insome cases, an IHS may respond to dynamically changing audio and/orvideo misalignment targeted to a participant while remaining resilientto audio transforms. An IHS may take an independent measure for eachaudio channel for each display. Audio channels may be grouped (e.g.,5.1, Stereo, Described video, Secondary Language, etc.) that havedifferent AV delays, audio codecs may be used with relative low qualitybut higher performance, watermarking may be used for audience ratings,and gain change and dynamic processing (compressor/limiter/AGC) may beused based on location of the participant, and physicalfingerprint/audio/video content enable incoming share.

8. Off-screen styles as mouse. An IHS may enable a single “style,”electronic pen, or pointer to be used locally to move object remotely orin another orthogonal display, thus reducing device diversity. A stylusmay have an IR/optical sensor in the tip so it may detect movement. Itmay be used it like a normal stylus on a screen (e.g., the horizontalscreen on a smart desk or table), and it may work as a mouse or pointingdevice with respect to an orthogonal screen (e.g., a vertical or walldisplay). The stylus can be moved across a horizontal desk surface forprecise mouse movements and, when lifted up and pointed somewhere, itmay accelerate the mouse movement. In some cases, pointing the device toa given display causes the stylus to be active with respect to thatdisplay, and inactive with respect to another.

9. Multi-Stylus synching with collaborative GUI for multi-user. In somecases, an IHS may identify a number of pens, number of participantsusing the pen and/or using pen and touch tiering. An overlapping GUI maybe displayed to multiple users. Touch sensors, display firmware, and/ora pen controller may change the color and or line shape of digital pento identify to other participants who is writing

10. Writing in VR/AR Space. In some cases, a stylus may be provided thatincludes an IR/optical sensor in the tip so it can detect movement. Acritical interaction is taking real object and take notes in AugmentedReality (AR) or Virtual Reality (VR) spaces. The pen may e used, forexample, with an AR glass and/or foldable OLED, and write in AR space.Feedback information to device and pen may be used withInternet-of-Things (IoT) technologies for information loading. Objectsmay be tracked and written to in AR space using a 3D camera andaccelerometer, for example with simple tags (e.g., “ok” or “not ok”).

11. Ambient light sensor and SAR fusion for display control. Scaling ofbrightness for two orthogonal displays is not natural for best GUIprovision. In some cases, two or more ambient light sensors (ALS) anduse proximity of the user (e.g., measuring specific absorption rate(SAR)) to scale brightness of the individual screen to uniform ambient.A dynamic brightness range may be used within the same display. For OLEDdisplay technologies, for example, this may be achieved by changing theblue intensity. For LCD display technologies, segmented LED from HDRtechnology may be used. In both cases, a brightness value may be scaledwith the GUI and distance from the intended user using ALS, 3D, and SAR.

It should be noted that the aforementioned features are not exclusiveand are provided for sake of illustration only, not necessarily in orderof importance. It should also be noted that any embodiment describedherein may provide any number of the aforementioned features, or none ofthem. Other features will also be apparent to a person of ordinary skillin the art in light of this disclosure, several of which are describedin more detail below.

In various embodiments, an IHS in the form of a smart display or smarttable may include a number of sensors that provide information useful inmanaging each participant's gesture, voice, and/or graphical userinterface(s) during an electronic meeting. Examples of sensors include:a 3D camera(s), proximity sensor(s) (e.g., a millimeter wave sensorhaving an antenna array, an ultrasonic sensor array, etc.), infrared(IR) sensor(s), Ambient Light Sensor(s) (ALS), etc.

In that regard, FIG. 1 is a diagram illustrating an example ofenvironment 100 where systems and methods for providing volumetrictracing for orthogonal displays in an electronic collaboration settingmay be implemented, according to some embodiments. As shown, environment100 may include, for example, a conference room, boardroom, meetingroom, or the like, where participants 104-107 are present.

In this implementation, environment 100 includes two IHSs implemented inthe form of a smart desk or table with horizontal display 101, and avertical display 102. That is, displays 101 and 102 are orthogonal withrespect to each other. Displays 101 and 102 may include a liquid crystaldisplay (LCD), an organic light emitting diode (OLED), a flat paneldisplay, a solid state display, or a cathode ray tube (CRT). Typically,gap 103 separates the two displays as shown, such that any in-roomparticipant (e.g., participant 104) may walk between the two displaysand at least partially block another participant's field of view and/orinterface.

Smart desk 101 includes 3D camera 108, ALS 109, and proximity sensors110. Similarly, display 102 includes 3D camera 111, ALS 112, andproximity sensors 113. In various implementations, cameras 108 and 111may include stereo triangulation cameras, a sheet of light triangulationcameras, structured light cameras, time-of-flight cameras,interferometry cameras, coded aperture cameras, or any other type of 3Dcamera. Proximity sensors 110 and 113 may include millimeter RF sensors,and ALS sensors 109 and 112 may include any suitable photo or lightdetector or the like. It should be noted, however, that other sensorsmay be employed by either IHS.

Participant 104, which in this case may be identified as a speaker orpresenter, is active and stands in a position such that his or her GUIis dynamically split across displays 102 and 101 in the form of areas114A and 114, respectively. Participant 105 is sitting at smart desk 101and his or her GUI is shown as area 118, which changes dynamically asphysical object 117 is placed in that surface, and which is restrictedto having a maximum radius 118, determined by the participant's size orreach. Meanwhile, participant 106 has his or her GUI 116 also displayedon screen 101. It should be noted that, in addition to GUI portions114-116, displays 101 and 102 may display content shared during theelectronic collaboration session (e.g., documents, graphs, charts,videos, presentations, etc.).

In this example, participant 107 is standing behind or next toparticipant 06.

Although participant 107 does not have a respective GUI on eitherdisplay, it may nonetheless be capable of interacting in a virtualmeeting by using gestures, audio commands, etc. In order to distinguishbetween each participants' different spheres of activity, sensor datafrom 3D camera 08 (or 111) may be used to determine a separation betweenparticipant 106 and participant 107 in areas 119 and 120, respectively.In addition, sensor data from proximity sensors 110 (or 113) may be usedto find another (different) separation between participant 106 andparticipant 107 in areas 121 and 122, respectively.

As will be understood by a person of ordinary skill in the art in lightof this disclosure, any business or non-business setting that requiresmeetings or collaboration may implement one or more aspects of thesystems and methods described herein. Additionally, aspects of theframework described herein may be expanded to other areas, such aseducational verticals for use in classrooms, or consumers for generalmeet-ups and/or reunions.

Virtual Collaboration Architecture

FIG. 2 shows Information Handling System (IHS) 200 configured to providevolumetric tracing for orthogonal displays in an electroniccollaboration setting. In various embodiments, IHS 200 may include a setprogram instructions that can be executed to cause the IHS to performany one or more of the methods or operations disclosed herein.

For purposes of this disclosure, an IHS may include any instrumentalityor aggregate of instrumentalities operable to compute, classify,process, transmit, receive, retrieve, originate, switch, store, display,manifest, detect, record, reproduce, handle, or utilize any form ofinformation, intelligence, or data for business, scientific, control,entertainment, or other purposes. For example, an IHS may be a personalcomputer, a tablet, a PDA/smartphone, a consumer electronic device, anetwork server or storage device, a switch router, wireless router, orother network communication device, or any other suitable device and mayvary in size, shape, performance, functionality, and price.

In a particular embodiment, IHS 200 can be implemented using electronicdevices that provide voice, video or data communication. In a networkeddeployment, IHS 200 may operate in the capacity of a server or as aclient user computer in a server-client user network environment, or asa peer computer system in a peer-to-peer (or distributed) networkenvironment. Further, while a single IHS 200 is illustrated, the term“system” shall also be taken to include any collection of systems orsub-systems that individually or jointly execute a set, or multiplesets, of instructions to perform one or more computer operations.

IHS 200 includes memory 209, one or more processing resources such as aCentral Processing Unit (CPU) 205 or hardware or software control logic,and operates to execute code. Additional components of IHS 200 mayinclude main memory 209, one or more storage devices such as staticmemory or disk drives 211. These memory devices 209 and 211 can storecode and data. In various embodiments, memory 209 and 211 may beimplemented using any suitable memory technology, such as static RAM(SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory,or any other type of memory.

Other components include one or more communications ports forcommunicating with external devices as well as various input and output(I/O) devices. I/O devices may include alphanumeric and cursor controldevices 260 such as a keyboard, a touchpad, a mouse, one or more videodisplay devices 225, display touchscreen(s) with touch controllers 230,and one or more cameras 270 including camera controllers. IHS 200 mayalso include one or more buses 218 operable to transmit communicationsbetween the various hardware components.

Again, IHS 200 may include one or more processing resources such as CPU205, Graphics Processing Unit (GPU) 206 that may or may not beintegrated with CPU 205, and related chipset(s) 208 or hardware orsoftware control logic.

In various embodiments, IHS 200 may be a single-processor systemincluding one CPU or a multi-processor system including two or more CPUs(e.g., two, four, eight, or any other suitable number). CPU(s) 205 mayinclude any processor capable of executing program instructions. Forexample, in various embodiments, CPU(s) 205 may be general purpose orembedded processors implementing any of a variety of Instruction SetArchitectures (ISAs), such as the x86, POWERPC®, ARM®, SPARC®, or MIPS®ISAs, or any other suitable ISA. In multi-processor systems, each ofCPU(s) 205 may commonly, but not necessarily, implement the same ISA.

IHS 200 may include several sets of instructions 221 to be run by CPU205, GPU 206, and any embedded controllers 220 on IHS 200. One such setof instructions includes Operating System (OS) 222 with an OS interface.

Example OSs 222 can include those used with typical mobile computingdevices such as Windows Phone mobile OS from Microsoft Corporation andAndroid OS from Google Inc. Additional sets of instructions in the formof multiple software applications 224 may be run by IHS 200. Theseapplications 224 may enable multiple uses of IHS 200, as set forth inmore detail below. Particularly, applications 124 may include programinstructions configured to provide a user interface for virtual meetingswith multiple shared displays, as discussed in several embodiments.

IHS 200 may operate as a standalone device or may be connected such asby using a network, to other computer systems or peripheral devices. IHS200 can represent a server device whose resources can be shared bymultiple client devices, or it can represent an individual clientdevice, such as an individual mobile personal computing system.

IHS 200 has network interface device 212, such as for a wirelesscellular or mobile networks (CDMA, TDMA, etc.), WIFI, WLAN, LAN, orsimilar network connection, enabling a user to communicate via a wiredor wireless communications network 213, such as the Internet. IHS 200may be configured with conventional web browser software. The webbrowser may include for example Microsoft Corporation's InternetExplorer web browser software, Firefox or similar such browsers to allowthe user to interact with websites via the wireless communicationsnetwork 213.

IHS 200 also includes one or more display devices 225 that may utilizeLCD, OLED, or other thin film technologies. Each display device 225 maybe capable of touch input via touch controller 230. Each display device225 has a display controller hub 235. The display controller hub 235 mayinclude control logic and software or access separate control logic andsoftware. Components may include a display controller or driver 237 anda backlight controller 240 for LCD thin film display technologies or abrightness controller for OLED/AMOLED technologies. The one or moreparts of the display controller hub 235 may be operated by or integratedwith one or more graphics processing units (GPUs) 206 such as those thatare part of the chipset 208. The display device 225 and one or moreparts of the display controller hub 235 may also be controlled by theembedded controller 220 of chipset 208. Each GPU 206 and displaycontroller/driver 237 is responsible for rendering graphics such assoftware application windows and virtual tools on the display device225.

IHS 200 may include one or more integrated camera systems 270 with oneor more camera drivers. Some cameras 270 may operate in accordance withnormal usage as a webcam or such as those that may be mounted on a backsurface for standard image taking with the IHS 200. Image taking orimage capture can refer to any type of camera operation whether stillimage capture, video, or other functions or for other purposes.

Alternatively, any one or all of camera(s) 270 may be used as part of avirtual I/O device hub 241 to capture images of gesture as part of thedisclosure embodiments described below. Camera(s) 270 may be part of orconnected to the virtual I/O device hub 241 for those having free spacegesture capture capabilities. For example, a camera operating as an I/Oinput device for gestures may also have a dual function and be used forimage capture purposes in other embodiments.

Some or all of the camera(s) 270 may be infrared (IR) cameras ornear-infrared cameras for use in low light or other difficult lightsettings or camera(s) 270 may operate in a typical visible lightspectrum. It should be noted that camera(s) mounted in other locationsmay similarly act as virtual I/O input devices with the virtual I/Odevice hub 241 to capture free space gestures according the systems andmethods disclosed in embodiments herein.

Additional components may be integrated in IHS 200 to work in connectionwith virtual I/O input hub 241. Such additional hardware may include animage sensor or proximity sensor, such as a low resolution IR or near IRcamera, to detect a user's hand location and movement. Additionally, asecond camera or IR system may act as a depth sensor to detect locationof a hand and orientation of palm and fingers in proximity to the cameraand IHS 200. For instance, a second camera may serve as additional imagesensor hardware. Cameras 270 may also be capable of capturing either twodimensional or three dimensional imagery of gesture actions and mayincorporate multiple image capture components or cameras 270 asnecessary to capture gesture images for interpretation to commands.Also, projection of a gesture boundary via a laser or LED projectionsystem may be employed as additional hardware integrated into IHS 200.Further description of the camera virtual I/O input device system isdescribed below as is additional description of additional integratedfunctionality and hardware for use in conjunction with the gesturedetection camera for I/O inputs to the IHS 200.

IHS 200 also has a system sensor module or component 250, includingmotion sensors 252, image sensors 224, sound sensors 256, proximitysensors 258, and Ambient Light Sensor (ALS) 260. Sensor system module250 is a sensor hub, or an accumulator device, that collects raw datafrom connected orientation sensors, and organizes and processes datareceived from the connected sensors. Such a sensor hub may be anindependent microcontroller such as the STMicro Sensor Fusion MCU aswell as other microcontroller processing systems. Alternatively, thesensor and fusion hub may be integrated into a core processing chipsetsuch as CPU systems for mobile devices as available fromIntel™Corporation or may utilize ARM Core processors that serve assingle or multiple core processors in alternative chipset systems. Thesensor hub may communicate with the sensors and chipset 108 via a busconnection such as an Inter-Integrated Circuit (I²C) bus or othersuitable type of multi-master bus connection. Sensor data may impactlocation of expected free space gestures and orientation of a gesturedetection boundary and gesture detection plane.

FIG. 3 is a block diagram of an example of virtual meeting system 300according to some embodiments. Generally speaking, virtual meetingsystem 300 may operate to manage an electronic meeting, presentation,conference, collaboration, work group, etc., while providing a userinterface for virtual meetings with multiple shared displays. In variousimplementations, IHS 200 of FIG. 2 may execute program instructions forproviding virtual meeting system 300. An Application ProgrammingInterface (API) may enable interaction between virtual meeting system300, device drivers, and other aspects of IHS 200 and its programinstructions.

Virtual meeting system 300 may connect to a bus of IHS 200 indirectlythrough network 213 and network interface device 212, or may connectdirectly to the network interface device 212 via the network channels(not shown). Virtual meeting system 300 may also connect directly to thebus of IHS 200.

Additionally or alternatively, virtual meeting system 300 may beimplemented in hardware. For example, a portion of virtual meetingsystem 300 and interface controller 302 may include hardware such as,for example, an integrated circuit (such as an Application SpecificIntegrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), astructured ASIC, or a device embedded on a larger chip), a card, or asystem (such as a motherboard, a system-on-a-chip (SoC), or astand-alone device). Virtual meeting system 300 may also have some orall parts to include software, including firmware embedded at a device,capable of operating a relevant environment of IHS 200. In some cases,virtual meeting system 300 may include a combination of the foregoingexamples of hardware or software.

In the example shown in FIG. 3, virtual meeting system 300 includesinput device component 301, interface controller component 302,three-dimensional (3D) camera component 303, Ambient Light Sensor (ALS)component 304, proximity sensor component 305, and other sensorcomponents (e.g., audio) 306. Each component may include a hardwaredevice, program instructions, drivers, APIs or any combination thereof.

In various embodiments, virtual meeting system 300 may acquire andprocess data regarding a three-dimensional image of one or morepresenter(s), participant(s), and/or object(s), as captured by cameras108 and/or 111 in FIG. 1, for example. In some embodiments, cameracomponent 303 works together with 3D mapping component 309 andperson/object recognition and tracking component 310 of interfacecontroller 302 to determine the 3D location of a participant and/or anon-participant object in a three-dimensional space, render a virtualthree-dimensional environment (e.g., including a three-dimensional imageof a participant and/or a non-participant object), and identify and/orrender a user interface plane on one or more displays (e.g., displays101 and 102).

In a further embodiment, interface controller 302 may receive andprocess data received from input devices 301 via input monitoring andcontrol component 312. Interface controller 302 may also receive andprocess data from camera component 303, ALS sensors 304, proximitysensor(s) 305, and other sensor(s) 306 via sensor monitoring, control,and fusion component 311. Data received from these various and sensorsand processed by sensor monitoring, control, and fusion component 311,and input monitoring and control component 312 may allow interfacecontroller 302 to perform operations such as, for instance, determiningthe three-dimensional location of a participant in an image captured atdifferent points in time and determining whether a participant'slocation has changed (e.g., using camera component 303 and proximitysensor(s) 305).

To this end, interface controller 302 may access a 3D image of aparticipant (or object) data describing the participant (or object)within the image. Interface controller 302 may also access a second 3Dimage of participants taken at a second instance in time and datadescribing the identification of the participant at the second instancein time. A series of 3D images may be displayed for a participant and/orobjects such that the movement of the three-dimensional image of aparticipant and/or object may operate as a video or an avatar substitutemay track movements of the three-dimensional image of a participant orobject.

Interface controller 302 may operate to display a graphical element oneach display in the location of the electronic meeting, or in any otherlocation (e.g., a remote location distinct from the environment 100),for example, using vertical display control component 307 and/orhorizontal display control component 308. In an embodiment, interfacecontroller 302 may operate to display a 3D image of a presenter on aremote observer's display. Vertical display control component 307 and/orhorizontal display control component 308 may also enable interfacecontroller 302 to control certain aspects and settings of each display(e.g., brightness, color, contrast, etc.).

Interface controller 302 may also access receive and process dataincluding: an electronic meeting's environment, a participant's locationand identification, an object's location an identification, a gesture orother command (e.g., voice) issued by a participant, a proximity of aparticipant to a display or smart desk, an ambient environment (e.g.,ambient lighting, temperature, etc.), profiles and preferencesassociated with a participant, etc.

Interface controller 302 may also operate to detect an input commandfrom a participant to manipulate a presentation graphical displayelement. In an embodiment, the input device component 301 may receive aninput command in the form of a keyboard entry, mouse entry, voicecommand, gesture command, or touch screen entry given in order to affectmanipulation of a presentation graphical display element, and softwareinstructions regarding manipulation of the electronic sharedpresentation graphical display element, as commanded. In anotherembodiment, a user input device may be a stylus, pen, or wearable cursorcontrol or other input device that may transmit location data withrespect to an electronic, shared presentation. Intersection of the planeof operation of the user input device and the electronic, sharedpresentation may indicate execution of commands or input to thepresentation application by one or more participants.

As noted above, interface controller 302 may be configured to identify acommand (e.g., a voice command or a gesture) made by a participant,and/or identify the intent of a participant in issuing that command, forexample, by analyzing video and/or audio cues, before, during, and/orafter the command is issued.

FIG. 4 is a block diagram of an example of gesture/intent detectionsystem 408 according to some embodiments. As illustrated, gesture/intentdetection system 408 represents a combination of aspects of 3D mappingcomponent 309, person/object recognition and tracking component 310,sensor monitoring, control, and fusion component 311, and/or inputmonitoring and control component 312 of interface controller 302 withinvirtual meeting system 300.

In an embodiment, gesture/intent detection system 408 may operate todetect and interpret gestures as I/O input. Such a gesture/intentdetection system 408 may be code run on CPU 205 or GPU 206 via chipset208. Alternatively, the gesture detection system 408 may be firmware orsoftware code instructions operating on a separate controller orprocessor. For example, gesture/intent detection system 408 may operateon or in close connection with vision system 401, which includes imagesensor 402 and camera 403.

Gesture/intent detection system 408 illustrates a subset of elements ofIHS 200 that enable free space gesture inputs. This subset includinghardware, firmware, and software elements for controlling a visionsystem 101, including camera 403 and image sensors 402. Gesture/intentdetection system 408 executes code in software or firmware on processorchipset(s) 208 for gesture detection and interpretation, and coordinatesgesture images and data. Alternatively, the code in software or firmwaremay be executed on a separate processor or microcontroller or may behard coded into an ASIC designed to perform some or all of theoperations of the gesture/intent detection system 408.

Generally speaking, gesture/intent detection system 408 is configured toidentify a user's hand and elements of the user's hand relevant to thegestures including a palm and one or more fingers that may performgestures. Gesture/intent detection system 408 interprets gesturemovements by a user's hand into commands for selection and manipulationof a cursor or elements displayed by OS 222 or application 224 runningon IHS 200. In an example, gesture/intent detection system 408 may alsomaintain a gesture detection plane and hand orientation. In anotherexample, gesture/intent detection system 408 may detect OS 222 orapplications 224 to be run on IHS 200 to determine what type of gesturesmay need to be supported for I/O input operations, such as cursorcontrol or the like.

In operation, camera 403 and image sensors 402 of vision system 401calibrate a detection zone and gesture detection plane. Additionally,camera 403 and image sensors 402 may detect and capture images ofgestures for input to virtual I/O hub 241. These images are buffered inimage buffer 404 and then provided to processor chipset 405 or otherprocessing device as needed by the gesture detection system 242.Gesture/intent detection system 242 may include several subcomponentsthat may be used to interpret the free space gesture images captured byvision system 401. Software application 407 includes programinstructions that determine the type of gestures supported by the activesoftware application running on IHS 200 at the time virtual I/O inputsare being used. For example, gesture/intent detection system 408 mayalso determine whether cursor control functions such as clicks, itemselection, scrolling, highlighting, drawing, writing, and other I/Ofunctions are available.

Gesture/intent detection system 408 may include a gesture classifier todetermine the type of gesture being detected. Examples of gesture typesavailable include orientation of the gesture such as whether gesturesare being received by a left hand or right hand or if the free spacegestures are being input in a hover position. Additional classificationmay include the type of user indication device being used to performfree space gestures for input. User indication devices may include auser's hand, pen or pencil, stylus or similar devices.

Classification of gestures may also depend on the category of gesturesbeing received. For example, one gesture detection category may bedetermined to be virtual mouse gestures based on the gesture imagescaptured. Detection of a virtual mouse click or the slide of a palm on agesture detection plane may indicate virtual mouse gestures. Anotherexample classification may include virtual trackpad inputs such as adetection of a finger touch and finger lateral movement on a gesturedetection plane.

Gesture/intent detection system 408 may include a signal module to takethe buffered images of block 404 and provide interpretation bysequencing images in preparation for rating and comparison with imagesstored in a gesture database. Such a gesture database may have datarepresenting multiple gestures available within a classification. Forexample, a right hand virtual mouse gesture classification may includepalm slides, virtual mouse clicks, virtual click and hold, virtualscroll wheel and other gestures. Additional database entries may includeclassifications for virtual touchpad gestures or pen/pencil/stylusgestures. A virtual touchpad gesture may include one finger touch,multiple finger touches, touchpad clicks, and virtual touches toactivation buttons as are understood by those of ordinary skill in theart as mimicking actual touchpad technology.

In some cases, variations to virtual mouse or virtual touchpad modes maybe enabled for the convenience of the user's or to improve upon thelimitations of the mimicked actual I/O devices. For example, additionalgestures may be implemented including variations from a typical mouse ortypical touchpad operation. This may include virtual clicks or virtualfinger touches with different fingers or multiple fingers or otherdetectable movements of the user's hand. A separate set of gestures maybe stored in the gesture database for each of the gestureclassification. For example, a similar set of virtual mouse gestures orvirtual trackpad gestures may be available for a user's left hand from aleft side mount camera or a re-oriented information handling system suchas a tablet or other mobile device. Yet another set of virtual mousegestures may be available for any position for gestures from eitherhand.

Gesture/intent detection system 408 may also include a rating andtrigger table, or the like. Once a gesture classification is determinedaccording to motion of a user's hand or other user indication device,the captured images of the gesture are compared with data for thegestures in that classification from the gesture database. A rating isassigned to the gesture based on its correspondence to the storedgesture from the gesture database. If the rating of the gesture motionfrom the images meets a threshold, the detected free space gesturetriggers an action in IHS 200. The action may be cursor control orselection of an element of a software application running on theinformation handling system. Other actions may include those allowed inthe detected application software running on IHS 200.

Examples of participants' gestures may include, but are not limited to,“need to speak,” “pointing,” and “voting.” When a participant indicatesheir need to speak during an electronic collaboration session, he or shemay employ user-definable gestures such as detection of raising theirhand, waving their hand, or cupping ears. Virtual meeting system 300 maythen determine that the participant needs a turn to speak or interact,and may provide a user interface to that participant to effect theparticipant's intent. As another example, virtual meeting system 300 mayidentify a displayed element that the participant would like toemphasize or highlight—e.g., by pointing to it—and that action may causea corresponding area on the shared display to be highlighted. Detectioncapabilities may also allow users' pinching and tapping -type gestures,among others.

In another embodiment, processor 405 may receive sensory inputs such asinfrared (IR) motion sensor component 406. The IR motion sensorcomponent 406 detects proximity of a hand or pen or other userindication device. IR motion sensor component 406 triggers the system tobegin to calibrate and detect gestures for input into the virtual I/Oand capture of gesture images for interpretation by the gesturedetection system in accordance with the disclosures herein. It isunderstood that variations on the several example disclosures herein arepossible for a gesture detection system and the virtual I/O hub devicesto calibrate, detect, capture and interpret free space gestures forinput to an information handling system.

In some cases, a laser projector (not shown) may be integrated with anIR camera used with IR motion sensor component 406. In an embodiment,the laser projection module shines a laser on a hand or other userindication at a plurality of points. For example, two or more laserpoints are shined. Using the laser points, the IR camera may detectmotion of the user's hand or other gesture indication device includingdirection vectors, velocity, acceleration and other dimensionalmovement. If the movement detected using the laser projection module andIR motion sensor component 406 meets a threshold of detected motion inthe free space gesture zone, system 408 initiates and the aforementionedoperations may be turned on. An example threshold may be a detectedvelocity of the hand or user indication that meets a threshold velocity(e.g., 1 ft/s). Velocity, acceleration, and/or direction vectors may beincluded as thresholds required to trigger the gesture detection systemvia virtual I/O hub 241.

The laser projection module may be integrated with the IR camera of IRmotion sensor component 406 or may be a separate device of IHS 200. Inaddition to the embodiment above, a laser projection module, whether ornot integrated with the IR camera, may also be used to project virtualworking boundaries of a virtual gesture detection plane.

In another embodiment, however, such a projection module may notnecessarily be laser light projection. For example, a LED or other lightsource may be used to approximately illuminate the virtual gesturedetection plane for free space gestures. In an embodiment, this lightmay be focused, for example variably focused, to approximate a virtualworking boundary for free space gestures.

In various embodiments, gesture/intent detection system 408 may employthe gesture recognition technology described with or without otherinputs from other sensors (e.g., using a fusion algorithm or the like)to discern a participant's intent. As noted above, gesture/intentdetection system 408 may include optical sensing technology and/orobject tracking technology, and may be implemented as a camera, athree-dimensional camera or body sensor, an infrared sensor, or the likeconfigured to detect body orientation, head orientation or gaze, and/orhand location/movement.

Examples of participants' intent detectable by gesture/intent detectionsystem 408 include intent to open, draw, move, erase content on adisplay, intent to point, intent to speak, intent to become a presenteror a non-presenting participant, etc. Additionally or alternatively,other features may be used to interpret gestures for presentationcontrol (e.g., next slide, previous slide, undo, redo, start blankpresentation, projector control such as volume, brightness, etc.). Itshould be noted, however, that these different intents and actions maybe extended to implement other intents or actions that are useful forvirtual meeting interactions, with additional input criteria that areappropriate for the new actions.

In operation, gesture/intent detection system 408 receives an indicationof body movement or motion. Examples of body movement include, but arenot limited to, face orientation, leaning toward a particular display,gaze orientation or direction, and hand or wristarticulation/location/movement. Gesture/intent detection system 408 mayalso determine whether a participant is facing a display, their ownhands, etc. For instance, gesture/intent detection system 408 may beconfigured to determine whether a participant's wrist is in a relaxed orflexible state. If so, then gesture/intent detection system 408 mayidentify the user's intent to draw when making a particular gesture,otherwise gesture/intent detection system 408 may detect theparticipant's intent when making the same gesture as an intent to erase.

As another example, gesture/intent detection system 408 may determinewhether the user is facing the audience (e.g., other in-roomparticipant(s), a broadcasting camera, etc.). If not, gesture/intentdetection system 408 determines if the user's wrist is rigid. If theuser's wrist is not rigid, gesture/intent detection system 408identifies the user's intent to point. Other non-limiting examples ofactions and the resulting detected intent are summarized below:

TABLE I Intended Action Classification Input/Attribute Draw Erase PointFace Facing display Facing display Facing orientation audience GazeLooking at Looking at Looking at hand hand audience Active ObjectDirecting Pen Activating/ Rotating/ (pen, totem, Sudden Move Directinglaser pointer) to a Point Wrist Flexible Rigid Rigid articulation

Coordination of vision system 401, gesture/intent detection system 408,and IR motion sensor component 406 of virtual I/O hub 241 may involvecontrol of camera 403 and gathering of gesture images for interpretationby the gesture/intent detection system 408 operating via processor 405,such as processor or chipset 208. Coordination occurs via instructionsrunning on or between elements of the processor chipset or an externalcontroller such as an MCU associated with the gesture/intent detectionsystem 408. This may include CPU, embedded controllers, and cameracontrollers, executing all or parts of instructions for capturing freespace gesture images for interpretation into information handling systemcontrol commands such as cursor control. CPU 205, embedded controller,MCU, camera controllers or other processors may coordinate via the IHS'sAPI.

The API coordinates the various elements of program instructions run onone or more processors in the chipset or controllers associated with theelements of vision system 401, their drivers, and the gesture/intentdetection system 408. These program elements may comprise the systems orperform the methods described herein that comprise the disclosed methodsand systems including several features of gesture/intent detectionsystem 408.

FIG. 5 is a block diagram of an example of proximity detection system504 according to some embodiments. As illustrated, system 500 representsa combination of aspects of person/object recognition and trackingcomponent 310, sensor monitoring, control, and fusion component 311,and/or input monitoring and control component 312 of interfacecontroller 302 within virtual meeting system 300.

In the embodiment shown, antenna array 501 may be used as one or more ofproximity sensor(s) 258, and may be implemented as a series of RadioFrequency (RF) antennas or antenna arrays 505 in combination withintegrated circuits or controllers. Proximity data sensed by antennaarray 501 may be received by processor 502 (e.g., CPU 205), whichexecutes program instructions in the form proximity application orsoftware library 503 to result in proximity detection system 504.

In some cases, antenna array 501 may include a phased array antenna at26, 60, 76 or 300 GHz, configured to directionally track a participant'sposition and movement 507 with respect to displays 101 and/or 102 singwireless, steerable power lobes 506A-D. For example, one or more sensorarrays such as antenna array 501 may be used around or outside theperimeter of displays 101 and/or 102. Using these frequencies, awireless power lob may be directed and switched at approximately a 1 to2 m distance from the array. For example, beam steering application mayreceive feedback from voice and ALS sensors based upon 60, 80, and 300GHz antenna and SAR. Sensors are evaluated and compared with a periodiccooperation in a real environment.

The power reflection coefficients at the air and skin interface at 60GHz for parallel and perpendicular polarized components shows that34-42% of the normal incident power is reflected at the skin surface atthat frequency, which may be higher at 300 GHz. By detecting SAR usingthe reflected power, a map of movement and location can be achieved.

In some cases, it may be critical to switch between the variousfrequencies. For example, based on obstacle and quality of signal,moving between 60 GHz and 300 GHz is found to be significantlyadvantageous.

In some embodiments, one antenna may be disposed at each corner ofdisplay 101, as part of a smart table. Additionally or alternatively,acoustic transducers may be used. For example, an acoustic sensor may beconfigured to track up to 2 or 3 persons (e.g., if 2 people are talkingin a conference room or discussing), and the visual sensor may beconfigured to support tracking when the acoustic sensor has difficulty.

FIG. 6 is a diagram of an example of switching system 600 according tosome embodiments. As illustrated, system 600 represents a combination ofaspects of person/object recognition and tracking component 310, sensormonitoring, control, and fusion component 311, and/or input monitoringand control component 312 of interface controller 302 within virtualmeeting system 300.

In the embodiment shown, frequency selection block 601 selects one ormore frequencies f₁-F_(N), for example, for operating an ambient lightsensor or the like. Frequency selection block 601 includes acorrespondence between different sensors such that a given frequency ofoperation for the ALS sensor corresponds to a second frequency ofoperation for another sensor which, in this example, is a proximitysensor such as sensor 258 and/or 501. In this example, multi-frequencyantenna array 602, signal strength analysis component 603, softwaredefined radio 604, sensor 605, and user interface (UI) file 606 may beused and/or executed in order to perform one or more of the variousoperations described herein. For example, components 601-606 may be usedto implement one or more sensor fusion operations.

Volumetric Tracing for Orthogonal Displays

In various embodiments, an electronic collaboration session may bemoderated by an IHS that records, analyzes, and determines a number ofparticipants in the room, identifies of one or more participants,identifies which participants are active and which are not, identifiesthe role of different participants in the session, identifies theposition and physical movement of each participant, identifies anyparticipant's gestures and/or intent, etc. This information may beobtained, for example, using a combination of 3D camera data andmillimeter wave proximity sensor data.

Based upon the aforementioned information, the IHS may provide a userinterface individualized to each participant, in some cases across aplurality of discrete displays.

Features of such an user interface may be selected based upon any of theforegoing information, and may change over time during a session asdifferent participants take different roles and/or physically movewithin the meeting room.

Moreover, features of the user interface may be selected based upondeterminations of areas, spheres, or volumes of activity for eachparticipant, again, potentially as the participant moves around theroom. Volumetric tracing and analysis may allow the IHS to distinguish,for example, which gestures are being made by either of two participantswhen one participant is in front of the other, what type of userinterface to offer each of the two participants (e.g., voice-only,gesture-only, voice and gesture-only, GUI-only, etc.)

Turing to FIG. 7, a flowchart of an example of method 700 for providingvolumetric tracing for orthogonal displays in an electroniccollaboration setting is depicted according to some embodiments. In somecases, method 700 may be implemented using virtual meeting system 300provided by IHS 200 disposed in environment 100. In this example, IHS200 also includes an implementation of gesture/intent detection system408, proximity detection system 504, and switching system 600.

At block 701, method 700 identifies a first characteristic associatedwith a given one of a plurality of participants of an electroniccollaboration session, at least in part, using an image sensing device.At bock 702, method 700 identifies a second characteristic associatedwith the given participant, at least in part, using a proximity sensingdevice.

In some cases, the image sensing device includes a 3D camera (e.g.,cameras 108 or 111 part of vision system 401). The first characteristicmay include a position of the given participant in three-dimensionalspace, an identity of the participant, a size or reach of theparticipant, a role of the participant in the electronic collaborationsession, a gesture made by the given participant, an intent of the givenparticipant, etc. Moreover, the proximity sensing device may include amillimeter RF wave sensor array (e.g., antenna array 501 part ofproximity detection system 504). The second characteristic may include aposition of the given participant relative to another participant or adisplay, a Specific Absorption Rate (SAR), etc.

At block 703, method 700 identifies a volumetric space based upon thefirst and second characteristics. In some cases, the volumetric shapedefines a three-dimensional area around a participant from within whichgesture or voice commands issued by that participant are recognizable bythe IHS, and outside of which gesture or voice commands issued by thatparticipant are not recognizable (or not executed) by the IHS.

At block 704, method 700 may provide a user interface to theparticipant, where the user interface has a set or one or more featuresselected, at least in part, based upon a characteristic of thevolumetric space. The characteristic of the volumetric space mayinclude, for example, a size, a position, or a shape of the volumetricspace.

The user interface may be a GUI, a speech recognition interface, or agesture recognition interface. The first set of one or more features mayinclude a size or area of the GUI on a display device. Additionally oralternatively, the first set of one or more features may include atleast one of: an image, an icon, a menu, a file, a folder, a script, asoftware application, or an Operating System (OS). Additionally oralternatively, the first set of one or more features may include anavailable voice or gesture command.

At block 705, method 700 may determine whether a change has beendetected to the first characteristic, the second characteristic, and/orto the volumetric space. Such change may be detected, for example, whena participant moves across the room, stands up, sits down, leans over,approaches a display, moves away from the display, etc. If no change isdetected (e.g., above a threshold), control returns to block 705.Otherwise, at block 706, method 700 modifies the set of featuresprovided in the user interface.

For example, in response to the change indicating that a givenparticipant's role in the electronic collaboration session has increased(e.g., from non-speaking to speaking participant, or from member tomoderator), the second set of features may be larger than the first setof features (e.g., larger or richer interface, exclusive commands,etc.). Conversely, in response to the change indicating that the givenparticipant's role in the electronic collaboration session hasdecreased, the second set of features may the smaller than the first setof features.

Additionally or alternatively, in response to the change indicating thatthe given participant has moved closer to a display, the second set offeatures may be larger than the first set of features, and in responseto the change indicating that the given participant has moved fartherfrom the display, the second set of features may be smaller than thefirst set of features.

For example, block 705 may determine that another participant movedcloser to the participant's volumetric space, and block 706 may reducethe volumetric space to disable one or more gestures available to theparticipant prior the other participant having moved. Additionally oralternatively, block 705 may determine that another participant movedfarther from the volumetric space, and block 706 may increase thevolumetric space to enable one or more gestures unavailable to theparticipant prior the other participant having moved.

In some cases, block 703 may identify another volumetric space foranother participant and provide another user interface to the otherparticipant, where other user interface has a second set of featuresaccessible to the other participant to the exclusion of the givenparticipant.

For instance, the first set or one or more features accessible to theparticipant to the exclusion of the other participant may include a GUI,and the second set of features accessible to the other participant tothe exclusion of the participant may include a voice or gesturerecognition interface. Additionally or alternatively, the first set orone or more features accessible to the participant to the exclusion ofthe other participant may include a gesture recognition interface, andthe second set of features accessible to the other participant to theexclusion of the participant may include a voice recognition interface.Additionally or alternatively, the first set or one or more featuresaccessible to the participant to the exclusion of the other participantmay include a touch interface, and the second set of features accessibleto the other participant to the exclusion of the participant may includea gesture interface.

In some scenarios, the other participant may be disposed behind theparticipant with respect to an image capture device. In that case, theIHS may receive two or more commands and determine that a given commandwas issued by the given participant based upon it having been detectedas originated from the volumetric space and that another command wasissued by the other participant based upon it having been detected asoriginated from the other volumetric space, at least in part, using aproximity sensing device.

As discussed above, the electronic collaboration session may use aplurality of distinct screens (e.g., 101 and 102) visible to theplurality of participants. In some cases, these screens may beorthogonal with respect to each other. For example, a first screen maybe part of a horizontal display device (which in turn may be part of asmart table or the like), and the second display may be part of avertical display device. Moreover, the horizontal and vertical displaydevices may be arranged with gap 103 in between that allows a givenparticipant 104 to stand between the display devices 101 and 102.

In various embodiments, providing the user interface may includedetermining that the given participant is standing in the gap andsplitting the user interface into a first portion and a second portion.For example, the first portion may be displayed on a first displaydevice with a first subset of features, and a second portion may bedisplayed on a second display device with a second subset of features.The first subset of features may include public information public withrespect to the plurality of participants (e.g., shared documents,presentation or playback commands, etc.), and the second subset offeatures may include private information with respect to the givenparticipant (e.g., personal documents, exclusive commands, etc.). Assuch, the user interface is dynamically distributed between the firstand second displays based upon the characteristic of the volumetricspace.

It should be understood that various operations described herein may beimplemented in software executed by logic or processing circuitry,hardware, or a combination thereof. The order in which each operation ofa given method is performed may be changed, and various operations maybe added, reordered, combined, omitted, modified, etc. It is intendedthat the invention(s) described herein embrace all such modificationsand changes and, accordingly, the above description should be regardedin an illustrative rather than a restrictive sense.

Although the invention(s) is/are described herein with reference tospecific embodiments, various modifications and changes can be madewithout departing from the scope of the present invention(s), as setforth in the claims below. Accordingly, the specification and figuresare to be regarded in an illustrative rather than a restrictive sense,and all such modifications are intended to be included within the scopeof the present invention(s). Any benefits, advantages, or solutions toproblems that are described herein with regard to specific embodimentsare not intended to be construed as a critical, required, or essentialfeature or element of any or all the claims.

Unless stated otherwise, terms such as “first” and “second” are used toarbitrarily distinguish between the elements such terms describe. Thus,these terms are not necessarily intended to indicate temporal or otherprioritization of such elements. The terms “coupled” or “operablycoupled” are defined as connected, although not necessarily directly,and not necessarily mechanically. The terms “a” and “an” are defined asone or more unless stated otherwise. The terms “comprise” (and any formof comprise, such as “comprises” and “comprising”), “have” (and any formof have, such as “has” and “having”), “include” (and any form ofinclude, such as “includes” and “including”) and “contain” (and any formof contain, such as “contains” and “containing”) are open-ended linkingverbs. As a result, a system, device, or apparatus that “comprises,”“has,” “includes” or “contains” one or more elements possesses those oneor more elements but is not limited to possessing only those one or moreelements. Similarly, a method or process that “comprises,” “has,”“includes” or “contains” one or more operations possesses those one ormore operations but is not limited to possessing only those one or moreoperations.

The invention claimed is:
 1. An Information Handling System (IHS),comprising: one or more processors; and a memory coupled to the one ormore processors, the memory including program instructions storedthereon that, upon execution by the one or more processors, cause theIHS to: identify a first characteristic associated with a given one of aplurality of participants of an electronic collaboration session, atleast in part, using an image sensing device; identify a secondcharacteristic associated with the given participant, at least in part,using a proximity sensing device; identify a volumetric space for thegiven participant based upon a combination of the first and secondcharacteristics, wherein the volumetric space comprises athree-dimensional area around the given participant from within whichgesture or voice commands issued by the given participant arerecognizable by the IHS, and outside of which gesture or voice commandsissued by the given participant are not recognizable by the IHS; providea user interface to the given participant, wherein the user interfacehas a set of one or more features selected, at least in part, based upona characteristic of the volumetric space; identify another volumetricspace for another participant and in response to the other participantbeing disposed behind the participant with respect to the image capturedevice, provide another user interface to the other participant, whereinthe other user interface has a second set of features accessible to theother participant to the exclusion of the given participant.
 2. The IHSof claim 1, wherein the first characteristic comprises at least one of:a position of the given participant in three-dimensional space, anidentity of the given participant, a size or reach of the givenparticipant, a role of the given participant in the electroniccollaboration session, a gesture made by the given participant, or anintent of the given participant, wherein the second characteristiccomprises a position of the given participant relative to anotherparticipant, and wherein the user interface comprises at least one of: aGraphical User Interface (GUI), a speech recognition interface, or agesture recognition interface.
 3. The IHS of claim 1, wherein thecharacteristic of the volumetric space comprises at least one of: asize, a position, or a shape of the volumetric space.
 4. The IHS ofclaim 1, wherein to identify the volumetric space, the programinstructions, upon execution, further cause the IHS to: determine thatthe other participant moved closer to the volumetric space; and reducethe volumetric space to disable one or more gestures available to theparticipant prior the other participant having moved.
 5. The IHS ofclaim 1, wherein to identify the volumetric space, the programinstructions, upon execution, further cause the IHS to: determine thatthe other participant moved farther from the volumetric space; andincrease the volumetric space to enable one or more gestures unavailableto the participant prior the other participant having moved.
 6. The IHSof claim 1, wherein the program instructions, upon execution, furthercause the IHS to: receive two or more commands; and determine that agiven command was issued by the given participant based upon it havingbeen detected as originated from the volumetric space and that anothercommand was issued by the other participant based upon it having beendetected as originated from the other volumetric space, at least inpart, using the proximity sensing device.
 7. The IHS of claim 1, whereinthe first set or one or more features accessible to the givenparticipant to the exclusion of the other participant comprises a GUI,and wherein the second set of features accessible to the otherparticipant to the exclusion of the given participant comprises a voiceor gesture recognition interface.
 8. The IHS of claim 1, wherein thefirst set or one or more features accessible to the given participant tothe exclusion of the other participant comprises a gesture recognitioninterface, and wherein the second set of features accessible to theother participant to the exclusion of the given participant comprises avoice recognition interface.
 9. The IHS of claim 1, wherein the firstset or one or more features accessible to the given participant to theexclusion of the other participant comprises a touch interface, andwherein the second set of features accessible to the other participantto the exclusion of the given participant comprises a gesturerecognition interface.
 10. The IHS of claim 1, wherein the programinstructions, upon execution, further cause the IHS to: identify adifferent volumetric space for the given participant based upon a changein at least one of the first or second characteristics during theelectronic collaboration session; and provide another user interfacehaving another set of one or more features selected, at least in part,based upon a characteristic of the different volumetric space.
 11. TheIHS of claim 10, wherein in response to the characteristic of thedifferent volumetric space indicating that the given participant's rolein the electronic collaboration session has increased, the set offeatures is larger than the other set of features, and wherein inresponse to the characteristic of the different volumetric spaceindicating that the given participant's role in the electroniccollaboration session has decreased, the set of features is smaller thanthe other set of features.
 12. The IHS of claim 10, wherein in responseto the characteristic of the different volumetric space indicating thatthe given participant has moved closer to a display, the set of featuresis larger than the other set of features, and wherein in response to thecharacteristic of the different volumetric space indicating that thegiven participant has moved farther from the display, the set offeatures is smaller than the other set of features.
 13. The IHS of claim1, wherein the electronic collaboration session uses a first screen partof a horizontal display device and a second screen part of a verticaldisplay device.
 14. The IHS of claim 13, wherein the user interface isdynamically distributed between the first and second displays based uponthe characteristic of the volumetric space.
 15. The IHS of claim 1,wherein the image sensing device comprises a three-dimensional (3D)camera, and wherein the proximity sensing device comprises a millimeterRadio Frequency (RF) wave antenna array.
 16. A hardware memory devicehaving program instructions stored thereon that, upon execution by anInformation Handling System (IHS), cause the IHS to: identify a firstcharacteristic associated with a given one of a plurality ofparticipants of an electronic collaboration session, at least in part,using an image sensing device; identify a second characteristicassociated with the given participant, at least in part, using aproximity sensing device; identify a volumetric space for the givenparticipant based upon a combination of the first and secondcharacteristics, wherein the volumetric space comprises athree-dimensional area around the given participant from within whichgesture or voice commands issued by the given participant arerecognizable by the IHS, and outside of which gesture or voice commandsissued by the given participant are not recognizable by the IHS; providea user interface to the given participant, wherein the user interfacehas a set of one or more features selected, at least in part, based upona characteristic of the volumetric space; identify another volumetricspace for another participant; and in response to the other participantbeing disposed behind the participant with respect to the image capturedevice, provide another user interface to the other participant, whereinthe other user interface has a second set of features accessible to theother participant to the exclusion of the given participant.
 17. Amethod, comprising: identifying a first characteristic associated with agiven one of a plurality of participants of an electronic collaborationsession, at least in part, using an image sensing device; identifying asecond characteristic associated with the given participant, at least inpart, using a proximity sensing device; identifying a volumetric spacefor the given participant based upon a combination of the first andsecond characteristics, wherein the volumetric space comprises athree-dimensional area around the given participant from within whichgesture or voice commands issued by the given participant arerecognizable by the IHS, and outside of which gesture or voice commandsissued by the given participant are not recognizable by the IHS;providing a user interface to the given participant, wherein the userinterface has a set of one or more features selected, at least in part,based upon a characteristic of the volumetric space; identifying anothervolumetric space for another participant; and in response to the otherparticipant being disposed behind the participant with respect to theimage capture device, providing another user interface to the otherparticipant, wherein the other user interface has a second set offeatures accessible to the other participant to the exclusion of thegiven participant.